Interaction between visual perception and mental representations of imagery and memory in the early visual areas

The relationship between mental representations based on external visual percepts (i.e., information held in short-term memory or via mental imagery) and the encoding of visual input remains unsettled. What stimulates this debate is the share of overlapping neural resources between visual short-term memory (VSTM), mental imagery and visual perception in the realm of the early visual cortex (EVC). This overlap raises a number of questions: how do the internal memory and imagery representations affect the perception of incoming visual information? What happens to imagery and VSTM abilities when cognitive resources need to be shared with the encoding of visual input? In short: how do visual memory/imagery and visual perception interact? This work addressed these questions by the use of behavioral paradigms coupled with transcranial magnetic stimulation (TMS) in situations where the encoding of the visual percept (measured via the tilt after effect (TAE) magnitude) happens either simultaneously or subsequently to holding information in VSTM/imagery. Therefore, when VSTM and the encoding of external input occurred concurrently, VSTM maintenance was found to inhibit visual encoding, reflected as a reduction of the TAE. Using TMS, it was shown that this inhibition takes place at the level of EVC. This reduction was found when the VSTM content matched the visual input, and when they were incongruent. However, when the encoding of external input occurred after VSTM maintenance phase had ended, VSTM maintenance was found to facilitate the former when the VSTM content matched the visual input. The subjective strength and the contrast of VSTM and mental imagery content (as reported by participants) affect visual detection of a briefly presented masked target. The reported visual contrast was positively associated with reporting target presence for both VSTM and mental imagery, in other words, inducing a more liberal bias. However, a differential effect was found for the subjective strength of the representations. Whereas the subjective VSTM strength was positively associated with the visual detection of the target, the opposite effect was observed for imagery. Finally, TMS applied at the EVC revealed a partial dissociation in the neural basis of VSTM and mental imagery by inducing delayed responses for the former selectively. Thus, while VSTM and mental imagery share neural resources, their neural mechanisms are partly dissociable at the level of early visual cortex.Many items constitute the visual environment. However, only those of relevance are maintained in mind. Once retained, the information will be kept active in the brain for a certain time. Therefore, the question that arises is, would this information, held in mind, have any simultaneous or subsequent effect on a real-time visual perception. This Work addressed this question by using simple tasks during which people where asked to memorize or imagine some shapes and subsequently or simultaneously perceive objects. Within the task, a non-invasive stimulation technique, known by transcranial magnetic stimulation (TMS), was applied. Therefore, participants received few magnetic pulses on their skull while performing the task. The results show that when information is held in mind while perceiving objects, the precision of the visual perception is reduced. This reduction is observed regardless of whether the percept is identical to that of the information held in mind. However, this is not the case when visual perception happens only after ceasing to hold information in mind. In this situation, a reduction of the precision of visual perception was found only when both information previously held in mind and the perceived to-be object were non-identical. The opposite pattern of results was observed when both were identical. Additionally, the processes of imaging and memorizing objects were found to have different effects on subsequent visual perception. Thus, whereas the subjective memory strength was positively associated with the visual detection of objects, the opposite effect was observed for imagery. Also TMS added to this discrepancy by affecting the speed of memory and imagery processes differently

Partial dissociation in the neural bases of VSTM and imagery in the early visual cortex Partial dissociation in the neural bases of VSTM and imagery in the early visual cortex-NC-ND license (http://creativecommons.org/licenses/by-nc-nd/4.0/)

a b s t r a c t Visual short-term memory (VSTM) and visual imagery are believed to involve overlapping neuronal representations in the early visual cortex. While a number of studies have provided evidence for this overlap, at the behavioral level VSTM and imagery are dissociable processes; this begs the question of how their neuronal mechanisms differ. Here we used transcranial magnetic stimulation (TMS) to examine whether the neural bases of imagery and VSTM maintenance are dissociable in the early visual cortex (EVC). We intentionally used a similar task for VSTM and imagery in order to equate their assessment. We hypothesized that any differential effect of TMS on VSTM and imagery would indicate that their neuronal bases differ at the level of EVC. In the "alone" condition, participants were asked to engage either in VSTM or imagery, whereas in the "concurrent" condition, each trial required both VSTM maintenance and imagery simultaneously. A dissociation between VSTM and imagery was observed for reaction times: TMS slowed down responses for VSTM but not for imagery. The impact of TMS on sensitivity did not differ between VSTM and imagery, but did depend on whether the tasks were carried concurrently or alone. This study shows that neural processes associated with VSTM and imagery in the early visual cortex can be partially dissociated

The mean (n=14) sensitivity (d’) and bias in Experiments 2 (panels A and B) and 3 (panels C and D).

<p>The Error bars indicate ± 1 SEM.</p

Timeline of an experimental trial.

<p>At the start of each trial, participants were shown a grating, which they needed either to hold in memory (VSTM condition) or to project as a mental image on the computer screen at fixation (imagery condition). During the maintenance, a masked grating was presented on 50% of trials; participants were asked to indicate whether or not they had perceived it. They were then asked to perform an orientation discrimination judgment based on the memory/imagery item; this involved indicating whether a test stimulus was tilted to the left or right relative to the memory/imagery cue. At the end of each trial, participants were asked to provide a rating of the strength of their memory or imagery on a 1-9 scale. In addition, they were asked to match the contrast of the grating held by memory/imagery to the exemplars presented on the screen. In Experiments 2 and 3, VSTM/imagery involved colored shapes (rectangle) while the visual target was the same as in Experiment 1 (see <a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0084827#pone-0084827-g001" target="_blank">figure 1B</a>).</p

Relationship between VSTM/imagery contrast/subjective strength and visual detection.

<div><p>The Error bars indicate SDs from which between-subjects variance has been removed [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0084827#B15" target="_blank">15</a>]. A) Mean (n=14) <i>visual </i><i>contrast </i><i>of </i><i>the </i><i>memory/mental </i><i>image</i> on the contrast response scale as a function of performance in the detection task For VSTM, contrast was significantly higher for “hits” than for other trial types. For imagery, contrast was significantly higher for “hits” and “false alarms” than for “misses” and “correct rejections”.</p> <p>B) Mean (n=14) <i>subjective </i><i>strength </i><i>of </i><i>the </i><i>memory/mental </i><i>image</i> as a function of performance in the detection task. For VSTM, subjective strength was significantly higher for “hits” than for other trial types. For imagery, subjective strength was significantly higher for “misses” and “false alarms” than for “hits” and “correct rejections”.</p></div

Mean subjective strength of VSTM in Experiment 2 (panel A) and imagery in Experiment 3 (Panel B) as a function of performance in the detection task.

<p>The Error bars indicate Error bars indicate ± 1 SEM.</p

The mean (n=14) sensitivity (d’) and bias in each condition.

<p>In the “BL” condition, the memory cue was not shown and no VSTM or imagery was required. In the “task” condition, participants were engaged either in VSTM or in visual imagery. A) Engagement in VSTM increased visual detection sensitivity relative to baseline trials but not significantly; no such effect was found for imagery. B) Engagement in VSTM decreased the bias; a similar but nonsignificant trend was present in the imagery condition. C) The mean memory accuracy in congruent and incongruent trials. Detection performance is plotted by the function of VSTM/imagery cue-target orientation congruency. The Error bars indicate SDs from which between-subjects variance has been removed [<a href="http://www.plosone.org/article/info:doi/10.1371/journal.pone.0084827#B15" target="_blank">15</a>].</p

Attention biases competition for visual representation via dissociable influences from frontal and parietal cortex

What mechanisms underlie the prioritization of neural representations of visually perceived information to guide behavior? We assessed the dynamics whereby attention biases competition for representation of visual stimuli by enhancing representations of relevant information and suppressing the irrelevant. Multivariate pattern analysis (MVPA) classifiers were trained to discriminate patterns of fMRI activity associated with each of three stimuli, within several predefined ROIs. Participants performed a change-detection task wherein two of three presented items flashed at 1 Hz, one to each side of central fixation. Both flashing stimuli would unpredictably change state, but participants covertly counted the number of changes only for the cued item. In the ventral occipito-temporal ROI, MVPA evidence (a proxy for representational fidelity) was dynamically enhanced for attended stimuli and suppressed for unattended stimuli, consistent with a mechanism of biased competition between stimulus representations. Frontal and parietal ROIs displayed a qualitatively distinct, more “source-like” profile, wherein MVPA evidence for only the attended stimulus could be observed above baseline levels. To assess how attentional modulation of ventral occipito-temporal representations might relate to signals originating in the frontal and/or parietal ROIs, we analyzed informational connectivity (IC), which indexes time-varying covariation between regional levels of MVPA evidence. Parietal-posterior IC was elevated during the task, but did not differ for cued versus uncued items. Frontal-posterior IC, in contrast, was sensitive to an item's priority status. Thus, although regions of frontal and parietal cortex act as sources of top–down attentional control, their precise functions likely differ